Detecting artificial multimedia content using deep network response analysis

ABSTRACT

Multiple trained AI models are tested using known genuine samples of respective multiple modalities of multimedia to generate versions of the multiple modalities of a given multimedia sample. Data for the multimedia and the multimedia sample are divided into the multiple modalities. Respective differences are computed between respective components of the multiple trained AI models to produce respective multiple difference vector, which are compared with corresponding baseline difference vectors determined in order to train the multiple trained AI models. The given multimedia sample is classified as genuine or altered using at least the comparison.

BACKGROUND

This invention relates generally to multimedia content and, morespecifically, relates to analysis of that content and use of theanalysis.

Improvements in Artificial Intelligence (AI) that can generate realisticmultimedia—images, videos, and text have enabled a variety of maliciousapplications such as creating fake news. A methodology for handlingartificial multimedia content is essential in several use cases such asidentifying and eliminating fake news, preventing ill effects caused byengineered media, and others.

The first step in addressing artificial multimedia is to be able todetect when elements of the media have signs of tampering. This is,however, not easily done.

SUMMARY

This section is meant to be exemplary and not meant to be limiting.

An exemplary embodiment is a method include testing multiple trainedartificial intelligence models using known genuine samples of respectivemultiple modalities of multimedia to generate versions of the multiplemodalities of a given multimedia sample. The data for the multimedia andthe multimedia sample are divided into the multiple modalities. Themethod includes computing, based on the testing, respective differencesbetween respective components of the multiple trained artificialintelligence models to produce respective multiple difference vectors.The method includes comparing the respective multiple difference vectorswith corresponding baseline difference vectors determined in order totrain the multiple trained artificial intelligence models. The methodalso includes classifying the given multimedia sample as genuine oraltered using at least the comparison.

Another example is an apparatus. The apparatus includes one or morememories having computer-readable code thereon, and one or moreprocessors. The one or more processors, in response to retrieval andexecution of the computer-readable code, causing the apparatus toperform operations comprising: testing multiple trained artificialintelligence models using known genuine samples of respective multiplemodalities of multimedia to generate versions of the multiple modalitiesof a given multimedia sample, wherein data for the multimedia and themultimedia sample are divided into the multiple modalities; computing,based on the testing, respective differences between respectivecomponents of the multiple trained artificial intelligence models toproduce respective multiple difference vectors; comparing the respectivemultiple difference vectors with corresponding baseline differencevectors determined in order to train the multiple trained artificialintelligence models; and classifying the given multimedia sample asgenuine or altered using at least the comparison.

A further exemplary embodiment is a computer program product. Thecomputer program product includes a computer readable storage mediumhaving program instructions embodied therewith. The program instructionsare executable by an apparatus to cause the apparatus to perform atleast the following: testing multiple trained artificial intelligencemodels using known genuine samples of respective multiple modalities ofmultimedia to generate versions of the multiple modalities of a givenmultimedia sample, wherein data for the multimedia and the multimediasample are divided into the multiple modalities; computing, based on thetesting, respective differences between respective components of themultiple trained artificial intelligence models to produce respectivemultiple difference vectors; comparing the respective multipledifference vectors with corresponding baseline difference vectorsdetermined in order to train the multiple trained artificialintelligence models; and classifying the given multimedia sample asgenuine or altered using at least the comparison.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a block diagram of an overview of one proposed methodology inan exemplary embodiment;

FIG. 1A is a flowchart of a process for artificial multimedia contentdetection in an exemplary embodiment, which is split into a trainingphase and testing phase and incorporates the methodology of FIG. 1;

FIG. 2 is a block diagram of a method for performing a first trainingstep, in accordance with an exemplary embodiment;

FIG. 3 is a block diagram of a method for performing a second trainingstep, in accordance with an exemplary embodiment;

FIGS. 4, 5, and 6 are illustrations of different testing steps for anexemplary proposed methodology, in accordance with an exemplaryembodiment; and

FIG. 7 is a block diagram of a system in which the exemplary embodimentsmay be practice, in accordance with an exemplary embodiment.

DETAILED DESCRIPTION

The following abbreviations that may be found in the specificationand/or the drawing figures are defined as follows:

AI Artificial Intelligence

AN Adversarial Network

CNN Convolutional Neural Network

GAN Generative Adversarial Network

GN Generative Network

LSTM Long Short-Term Memory

NN Neural Network

RNN Recurrent Neural Network

The word “exemplary” is used herein to mean “serving as an example,instance, or illustration.” Any embodiment described herein as“exemplary” is not necessarily to be construed as preferred oradvantageous over other embodiments. All of the embodiments described inthis Detailed Description are exemplary embodiments provided to enablepersons skilled in the art to make or use the invention and not to limitthe scope of the invention which is defined by the claims.

As previously stated, there is a great demand and need for technology indetection of forged media. This media is referred to herein asartificial multimedia. Artificial multimedia is any multimedia that hasbeen modified from an original version, often with the intention ofbeing undetectable as artificial. The first step in addressingartificial multimedia is to be able to detect when elements of the mediahave signs of tampering. With the advancements in technologies such asgenerative adversarial networks, it has become increasingly easier tocreate convincingly forged media that is difficult to identify even uponhuman inspection, unless the observer has prior knowledge of thetampering.

In an exemplary embodiment, proposed methodology classifies eachcollection of related multimedia content as genuine or altered. Forinstance, an exemplary method leverages modality-specificrepresentations and combines the representations at a high-level topredict if a set of multimedia is genuine or altered. These techniquesmay also enable further restorative/suppressive procedures to be appliedon the altered multimedia content.

The approach is scalable to any number and/or kind of modalities.Further, the exemplary method works independent of particular types ofalterations and is generalized. Performance on different alterationswill depend on the manifestation of these alterations in the chosenindividual representation space. Additionally, exemplary methods hereincan be extended to problems beyond fake news detection, such asdetecting poor quality media created for applications like dataaugmentation.

By contrast, existing approaches are either specific to only onemodality, or specific to particular types of alterations ormanipulations.

With respect to an exemplary embodiment, a combination ofmodality-specific network architectures is leveraged, along with amulti-modal aware combination framework, to extract discriminativefeatures (e.g., as vectors) from a given set of related multimediacontent. A classifier framework can use these features to categorizemultimedia into genuine or altered categories. This classifier frameworkmay be applied to static or sequence data (e.g., a set of static data inan order) by adapting the classifier strategy and algorithm as per themanifestation of the methodology.

This is illustrated in part by FIG. 1. Referring to FIG. 1, this figureis a block diagram of an overview of one proposed methodology 100 in anexemplary embodiment. The methodology 100 is implemented by a computersystem 710 in an exemplary embodiment, and this computer system 710 isdescribed in FIG. 7. There are three modalities 1 110-1, 2 110-2, and 3110-3, each of which inputs to a corresponding GAN module 1 120-1,120-2, and 120-3. Each modality 110 corresponds to a component ofmultimedia, such as voice or other audio, images, or video.

Before proceeding with additional description of the exemplaryembodiments, it is helpful to define some terms. The term video as usedherein is assumed to have an audio component and a visual component. Theaudio component could, for instance, be an audio channel, and the visualcomponent could be a video channel. The audio component could be anaudio signal, and the visual component could be a corresponding sequenceof visual frames. Visual frames may also be referred to as video frames.

Returning to description of FIG. 1, each GAN module 120 produces acorresponding vector set 1 130-1, 2 130-2, and 3 130-3. Each vector set130 can be considered to correspond to features of the corresponding GAN120 that can then be used to distinguish between media. Thus, thevectors 130 are also referred to herein as feature vectors 130. Each GAN120 could have multiple vectors, though these may also be combined intoa single vector. For ease of reference, these are described herein as asingle vector for each GAN 120, though this is not a limitation of theteachings herein. The vectors 130 correspond to components 115 for eachof the GANs 120, and the components 115 correspond to certain (e.g.,key) nodes and/or layers in the corresponding GAN 120. This is describedin more detail below.

Concerning the GAN modules 120, a GAN module uses two different NNs,each pitted against the other. The two networks are the generativenetwork (GN) 121 and the adversarial network (AN) 122, shown only forGAN module 1 120-1, although the other GAN modules 120-2 and 120-3 usesimilar structures. In the examples herein, the role of the adversarialnetwork is limited to the training phase, where the adversarial network122 tries to continuously force the generative network 121 to do betterand create better images (or other elements, such as audio or video).For embodiments herein, the trained generative part of the network,illustrated as GN 121, is utilized and therefore represented as the “GANmodule” 120.

The combination framework 140 operates on the vector sets 130 and formsdiscriminative vectors 145, which are unified vectors used by theclassifier 150 to determine an output 160 of whether an input to thesystem is genuine or altered. This output 160 may be, e.g., from zero(e.g., genuine) to one (e.g., altered), e.g., or some value betweenthese. The output may also be “genuine” or “altered” or the like,depending upon implementation.

With respect to the combination framework 140, there are multiplepossible manifestations of this. In one manifestation, the combinationframework 140 can be expressed as a weighted average of the independentfeature vectors where the weights can be set at a modality-level orlearned from training data. In another manifestation, the combinationframework 140 may be a neural network that is trained alongside thefinal classifier or includes the classifier as a layer within its ownarchitecture. In yet another manifestation, the combination framework140 may rely on statistics or derived variables extracted from theindependent feature vectors to compute the final feature vector.Additionally, one or more combination frameworks 140 may be combined tocreate an ensemble-based combination strategy.

While GANs and their corresponding neural networks are mainly describedherein, the combination framework 140 and models 120 may involve anykind of artificial intelligence (AI) models or techniques. For instance,machine learning, deep learning, or natural language processing, or somecombination of these, models and techniques might also be used. Morespecifically, to illustrate, convolutional and/or sequentialautoencoders may be used to generate signatures for artificialmultimedia, and techniques such as boosting or group sparse classifiersmay be utilized to build the combination framework.

An exemplary proposed methodology is split into two training steps andthree testing steps, although other number and types of steps may beused. This is illustrated by FIG. 1A, which is a flowchart of a trainingphase and testing phase, and which incorporates the block diagram ofFIG. 1, in an exemplary embodiment.

In FIG. 1A, this shows a flowchart of a process 190 for artificialmultimedia content detection in an exemplary embodiment, which is splitinto a training phase 162 and a testing phase 170. This process 190 mayalso be implemented by the computer system 710. The training phase 162is used to train the elements in the methodology 100 and to create andtrain the classifier 150 that is then used in the testing phase 170.

In block 164 of the training phase, the computer system 710 determinesbaseline signatures of AI (artificial intelligence) model components 115(e.g., key nodes and/or layers) in the modality-specific generativearchitectures. The architectures are the GANs 120 in an example. Thebaseline signatures correspond to the components 115. One example ofblock 164 is referred to as training step 1, which is illustrated inFIG. 2.

In block 168, the computer system 710 determine comparison vectors 130,using the determined signatures, and trains a set of classifiers 150using the comparison vectors. While it is possible to keep each vector130 for each modality 110 separate, and to perform an analysis based onthese separate vectors, the examples herein combine the vectors 130using the combination framework 140. This results in a set ofclassifiers 150, which may then be used to determine a resultant output160 of genuine or altered. One example of block 168 is referred to astraining step 2, and is illustrated by FIG. 3.

Once the training phase is complete, the testing phase 170 may then beused to determine whether a given sample of media is genuine or altered.The testing phase 170 is in the example of FIG. 1A split into threetesting steps.

In block 174, the computer system 710 reconstructs individual componentsof a given media using the trained modality-specific generativearchitectures. The given media is a multimedia sample to be tested aseither genuine or altered. This is testing step 1, one example of whichis illustrated in FIG. 4. In block 176, the computer system 710 computesdifferences in signatures of AI model components 115 (e.g., key nodesand/or layers) between given media and previously trained genuine mediato create difference vectors (as compared to comparison vectors) fordifferent modalities. That is, the previously stored vectors 130 arecompared with vectors 130 determined during the reconstruction using agiven media. This is testing step 2, an example of which is illustratedin FIG. 5.

In block 178, the computer system 710 combines and classifies vectors todetermine whether the given media is genuine or altered. This is testingstep 3, illustrated in an example in FIG. 6.

Although the emphasis herein is placed on describing exemplaryembodiments using a single given media for testing, it is possible totest multiple media. For instance, multiple media may be tested seriallyin an exemplary embodiment. In other words, it is possible to run thetests for multimedia (MM) 1, determine whether MM 1 is genuine/altered,run the tests for MM 2, determine whether MM 2 is genuine/altered, andthis is repeated for each of the multiple media.

Now that an overview has been provided, more detail is provided.

Turning to FIG. 2, this is a block diagram of a method for performing afirst training step, corresponding to block 164 of FIG. 1A, inaccordance with an exemplary embodiment. In block 210, a set ofpretrained modality-specific generative architectures (i.e., GAN modules120) is selected. For instance, CNNs could be chosen for images or videoframes, and RNNs might be chosen for audio or text. These are chosenfrom a pool of available generative architectures 120 that have beenpretrained using other pertinent datasets. The term “other” is used hereto indicate that, in this exemplary embodiment, the dataset(s) used forthe testing process is not going to be used for the process in FIG. 2.The other pertinent dataset(s) can be other different dataset(s) thatneed not be a part of the current dataset(s) or similar data to whichthe algorithm for evaluation/inference will be applied. The dataset ispertinent to the modality 110, such as images, audio, video, or thelike.

As additional detail, the selection of the appropriate GAN architecturesmay include selection of the number and type of layers, how they areconnected, what loss function and hyperparameters to use, and the like.The selection is based on the type of fake multimedia that will bedetected. For example, if one wants to verify the organic (i.e.,genuine) authenticity of videos that primarily deal with peopletalking/giving interviews, it is possible to select GAN architecturesthat are shown to have high performance in generating human speech andhuman face images.

Then, these architectures (e.g., just the skeletons) of thesemodality-specific GANs are used, and potentially also their pre-trainedweights (e.g., which are a result of training using open source datasetsthat they report their performance on). Optionally, if possible, one maywant to fine-tune (e.g., take the existing weights and tweak them usingother database(s)) them, using more task-specific databases ifavailable.

For example, fine-tuning model selection parameters using the specificapplication domain may be used to further refine the set of chosenarchitectures to best suit the domain. See block 220. This is becausecertain architectures work better for a given type of data than others.For example, as described above, CNNs might work better (e.g., relativeto RNNs) for images or video frames, and RNNs might work better (e.g.,relative to CNNs) for audio or text.

Using the default parameters of the pretrained network, the baselinebehavior patterns (also referred to as signatures) of key nodes/layersmay be determined in the generative architectures. See block 230. Theterms “behavior pattern” and “signature” are used interchangeably, andeach of these is what is referred to as the vector(s) of the activationstate(s) of each key node/layer in the network. Key nodes and/or layerscan be determined for each architecture independently, e.g., byobserving which of them showcase consistently high contribution (e.g.,high activation) when the architecture generates synthetic images (as anexample of a modality 110) with desired properties. The desiredproperties can be defined based on the task and domain of the media.

In more detail, perform GAN analysis/dissection may be performed usingmethodologies that can help quantify the contribution of differentlayers/nodes in the respective GANs when these networks generate theirbest output. This is effectively training step 1, where it is learnedhow the GANs behave. One aim of this first training step is to learn themost important components of each GAN. In an example case presentedbelow, there are two modalities 110 (a visual component and an audiocomponent of video) and therefore two GANs 120 that will be used toperform this step. Once these components 115 and their correspondingsignatures are determined for each GAN, this concludes training step 1.

Turning to FIG. 3, this figure is a block diagram of a method forperforming a second training step (see block 168 of FIG. 1A), inaccordance with an exemplary embodiment. This method is performed afterthe method in FIG. 2 has been performed.

Now, since it is known which components (layers/nodes) are important ineach GAN, one can begin applying them to reconstruct (e.g., in the imagespace) organic (i.e., genuine) and synthetic (i.e., altered) samplesthat closely resemble the target use case. In one example, it would beorganic videos and synthetic videos (e.g., where synthetic videos may begenerated using the organic videos and computer vision/speech processingtechniques). Even a relatively small number of labeled videos should besuitable (where “labeled” means the video is labeled as either genuineor synthetic).

In block 310, given a dataset of genuine and altered related multimediacontent, where the ith sample of n related media items from a dataset isdenoted as {m_(i1), m_(i2), . . . , m_(in)}, an attempt is made torecreate both the genuine and altered multimedia content using relevantnetwork architectures, such as the GANs 120 that have been selected forthe corresponding modalities. For instance, as previously described,CNNs could be chosen for images or video frames, and RNNs might bechosen for audio or text. This attempt creates a reconstruction error.

In additional detail, the ith notation denotes one sample from thedataset where each sample contains n media items. If the example isvideo, then n=2 (i.e., an audio component and a visual component) and ina dataset of 100 such media, the value i can range from 0 to 99. Eachitem will have i1 and i2, denoting the video and audio components (thetwo related media items) of that particular video in the dataset.

The recreation will never be exact and have some recreation (also termedreconstruction) error. While one can try to reduce the error below atolerance/threshold, in many cases, there will typically not be zeroerror. Therefore, block 310 says that there is an “attempt” to createthe best recreation that can be recreated.

The relevant network architectures are modality-specific relevant GANmodules 120. A GAN module 120 trained on images might not be useful toreconstruct a sound signal, for example, and vice versa.

Additional training data for the altered class may be generated usingfake generation techniques and/or creating deliberately tampered datausing media-specific methods. For instance, mis-syncing the audio trackof a video by different offsets can generate different altered versionsof the same video.

As samples are reconstructed using these GANs, monitoring and recordingis performed of the behavior of the components 115 (e.g., keynodes/layers) and the pattern(s) (as vector(s) 130) that exist in thedifferences between their behavior on organic and synthetic samplesbecome the feature vectors 130. The reason this is performed for thesecomponents 115 and not for all nodes/layers in the network is to keepthe feature dimension reasonably low and high impact. For example, if itis observed that nodes 50-100 in layer 3 and nodes 25, 26, and 27 inlayer 5 are consistently taking different value ranges when attempts aremade to reconstruct organic videos with the GAN versus synthetic videos,then these can be arranged as a vector in a fixed sequence and made oneof the feature vectors. In an exemplary embodiment, this observation isby a human but computed and identified programmatically—such as by apython script or other programming. It is also possible for this to beentirely

In light of this, in block 320, once the reconstruction error isminimized to be below (e.g., meet) a tolerance threshold, the internallayer outputs of these architectures are dissected. In particular, thedissection views at least some or all of the same key nodes/layers as inblock 230. In an exemplary embodiment, the determination of thecomponents (e.g., key nodes/layers) may be based on the ranking ofcomponents, which is based on how often the components take largervalues (e.g., high activation means high contribution to the finalresult) when the generative network outputs a good quality (e.g., veryrealistic) synthetic result.

In block 330, the response of these key nodes and/or layers are comparedfor genuine media and for the altered media with the baseline patterns.This operation creates comparison vector(s).

Once the feature vectors are in place, one can combine them using acombination framework. This can be implemented in different ways, suchas concatenation, using a deep/shallow neural network, and/or usingfusion or other techniques. This obtains a final vector for themultimedia content. This final vector is, in an exemplary embodimentbelow, considered to be a mean genuine feature vector. Aftercombination, in block 340, a set of classifiers are trained on theresulting comparison vector(s). Any type of classifier may be trained ontop of the feature vectors 130.

To generate the feature vectors 130 for test-time content, such as amultimedia sample to be tested as being genuine or altered, the meanorganic (genuine) media response is stored (see block 34) as referencefeature vectors 130 for future usage. This is GAN-specific, so in in theexample case presented below, there will be two sets of this, one setfor an audio component and one set for a visual component. Thisconcludes training step 2.

At this point, the training phase 162 has been described. Once thetraining phase is complete, the testing phase may be performed. As anoverview, during testing, stored classifiers are leveraged, and thesteps as illustrated in the testing phase are repeated, but using agiven multimedia sample. The given multimedia sample is reconstructed.During this process, the behavior of the components 115, e.g., the keynodes/layers for the corresponding GANs 120 is recorded. The behavior iscompared with the mean recorded behavior for each GAN that was storedafter training step 2. A difference vector is obtained, this is passedto the trained classifier 150, and the output 160 that denotes whether aparticular modality is detected as synthetic (i.e., genuine) or alteredis determined. This concludes the flow of the overall algorithm fromtraining to testing.

With respect to testing, FIG. 4 is an illustration of a first testingstep, see also block 174 of FIG. 1A, which is where the computer system710 reconstructs individual components of a given media using thetrained modality-specific generative architectures. That is, this figureillustrates reconstruction of individual components of the givenmultimedia sample 410, which is a video having a visual component of thevideo channel 110-1 and an audio component of the audio channel 110-2.This reconstruction is performed such that a reconstruction error (e.g.,such as perceptual loss) is below the tolerance threshold. See block490. Here, the modality-specific architectures of the GANs 120-1 and120-2 are leveraged in order to perform the best reconstruction. Beyondmodality, generative architectures may be fine-tuned further for thespecific type of data, e.g., faces versus landscapes so that theoptimization process is more efficient. This has been previouslydescribed.

In FIG. 4, a multimedia sample 410 is provided, and this has twomodalities 110-1, a video channel, and 110-2, an audio channel. As anoverview and for an exemplary embodiment, this uses the generativenetwork of the selected GAN architecture and minimizes the perceptual(in a corresponding image/visual space) error between a generated imageand an original image. Similar functions are performed for the audio.

An example of an image of the video channel is illustrated as reference420, and an example of a portion of audio of the audio channel isillustrated as reference 430. In this example, the GAN 120-1 is aCNN-based GAN for visual data generation, and the GAN 120-2 is anRNN-based GAN for audio data generation. It is noted that the RNN-basedGAN 120-2 has Long Short-Term Memory (LSTM) cells as part of thenetwork. The computer system 710 attempts to reconstruct the multimediasample 410 by generating the video channel 460-1 (one part of which isillustrated by reference 495) and generating the audio channel 460-2(one part of which is illustrated by reference 470). A metric is usedfor each of these to determine how well the generated version 460-1,460-2 corresponds to the versions 110-1, 110-2 in the sample 410. Inthis example, the computer system 710 performs a reconstruction errordetermination 490 to minimize loss 480-1 for the video channel (e.g.,using a first tolerance threshold) and to minimize loss 480-2 for theaudio channel (e.g., using a second tolerance threshold).

Turning to FIG. 5, this is an example of a second testing step. Seeblock 176 of FIG. 1A, where the computer system 710 computes differencesin signatures of AI model components 115 (e.g., key nodes and/or layers)between given media and previously trained genuine media to createdifference vectors (as compared to comparison vectors) for differentmodalities. That is, the previously stored vectors 130 are compared withvectors 130 determined during the reconstruction using a given media.

For FIG. 5, the difference is determined in the signatures of the AImodel components (e.g., the most prominent layers and/or nodes) in thegenerative architectures (e.g., GANs 120) between the given multimediasample(s) and previously processed genuine/unaltered samples. This canbe seen as the network state analysis 510-1 of the GAN 120-1 for videodata generation, which results in input video feature vector 520-1comprising input audio features. The input video feature vector 520-1 isan example of a (e.g., testing/input) feature vector 130. This can alsobe seen as the network state analysis 510-2 of the GAN 120-2 for audiodata generation, which results in input audio feature vector 520-2comprising input audio features. The input audio feature vector 520-2 isan example of a (e.g., testing/input) feature vector 130. Each networkstate analysis 510 is a combination that uses corresponding ones of thesignatures learned during training (e.g., see block 164 of FIG. 1A) andthe network state obtained after the reconstruction showed in FIG. 4.

The differences may be computed using various methods and a differencevector 550 is created for each modality. Specifically, the input videofeature vector 520-1 is compared (block 540-1) with a previously storedfeature vector 130, shown as a mean genuine video feature vector 530-1from training. Mean genuine feature vectors 530 are one exemplary outputof step 168 of FIG. 1A. The comparison results in the input videofeatures difference vector 550-1. The input audio feature vector 520-2is compared (block 540-2) with a previously stored feature vector 130,shown as a mean genuine audio feature vector 530-2 from training. Thecomparison results in the input audio features difference vector 550-2.

Turning to FIG. 6, this is an example of a third testing step, where acomputer system 710 combines and classifies vectors to determine whetherthe given media is genuine or altered. In FIG. 6, the two differencevectors 550-1, 550-2 of independent modalities are passed to thecombination framework 140. The combination framework 140 provides aunified feature vector, shown as input multimedia feature vector 610,for the entire set. The classification framework 620 classifies thesample set as genuine/altered 160 using the trained classifier 150 (seeFIG. 1). This example shows a two-dimensional space where featurevectors mapped into the area 622 are considered to be altered, whereasfeature vectors mapped into the area 623 are considered to be genuine,and the line 621 distinguishes between the two spaces 622, 623. Notethat this is for ease of exposition only, as normal feature vectorswould be N-dimensional, where N is typically greater than two. Thisdetection result 160 is output in block 630, which may be output to anetwork and/or a display, for instance. The output in block 630 may takemany forms, such as an integer (e.g., zero=genuine, one=altered), a realnumber (e.g., values under 0.5 are genuine, values about 0.5 arealtered), text (e.g., “genuine” or “altered”) or the like.

Turning to FIG. 7, this figure is an example of systems 700 in which theexemplary embodiments might be practiced. The computer system 710comprises the following: one or more processors 720; one or morememories 725; network (N/W) interface(s) (I/F(s)) 745, one or moretransceivers 730 having one or more antennas 728, and user interface(I/F) circuitry 765, all interconnected through one or more buses 727.The computer system 710 may also include one or more user I/F elements705, such as camera(s), audio device(s), sensor(s), display(s),keyboard(s), and/or other input device(s) (such as trackballs, mice,touch screens, and the like). The one or more transceivers 730 include areceiver (Rx) 732 and a transmitter (Tx) 733.

The one or more buses 727 may be address, data, or control buses, andmay include any interconnection mechanism, such as a series of lines ona motherboard or integrated circuit, fiber optics or other opticalcommunication equipment, and the like. The one or more memories 725include computer program code 723. The computer system 710 includes acontrol module 740, comprising one of or both parts 740-1 and/or 740-2,which may be implemented in a number of ways. The control module 740implements the detecting artificial multimedia content using deepnetwork response analysis as previously described. The control module740 may be implemented in hardware as control module 740-1, such asbeing implemented as part of the one or more processors 720. The controlmodule 740-1 may be implemented also as an integrated circuit or throughother hardware such as a programmable gate array. In another example,the control module 740 may be implemented as control module 740-2, whichis implemented as computer program code 723 and is executed by the oneor more processors 720. For instance, the one or more memories 725 andthe computer program code 723 may be configured to, with the one or moreprocessors 720, cause the computer system 710 to perform one or more ofthe operations as described herein.

The computer system 710 may communicate with one or more wired and/orwireless networks via one or both of wireless link 778 or wired link777. Another computer system 790 may also be used, comprising a display795, a browser 796, and a UI 797. Users 701 may interact with one orboth of the computer system 710 (user 701-1) or computer system 790(user 701-2).

In one example, the computer system 710 is an on-premise computersystem, where a user 710-1 is on the same premises as the computersystem 710. The computer system 710 can communicate with internal andexternal networks 797. In another example, the entire system 700 may beon-premises, such that a user 701-2 uses a front-end computer system 790to connect via the network(s) 797 with a back-end computer system 710.The back-end computer system 710 has the control module 740 thatimplements the detecting artificial multimedia content using deepnetwork response analysis, and creates output to display the UI 797(e.g., within the browser 796) on the display 795.

As another example, the system 700 may be in separate locations, and theuser 701-2 can connect via the one or more networks 797 (e.g., theInternet) to the computer system 710, which then has the control module740 that implements the detecting artificial multimedia content usingdeep network response analysis, and creates output to display the UI 797on the display 795. The computer system 710 may be implemented in thecloud for instance, and the visualization could be offered as a service.The computer system 710 could also be a server and the computer system790 a client, as another example of a possible implementation.

There is a great demand and need for technology in detection of forgedmedia, and the instant techniques help with detection of this media. Thetechniques herein can be extended to problems beyond fake newsdetection—such as detecting poor quality media created for applicationslike data augmentation.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

What is claimed is:
 1. A method, comprising: testing multiple trainedartificial intelligence models using known genuine samples of respectivemultiple modalities of multimedia to generate versions of the multiplemodalities of a given multimedia sample, wherein data for the multimediaand the multimedia sample are divided into the multiple modalities;computing, based on the testing, respective differences betweenrespective components of the multiple trained artificial intelligencemodels to produce respective multiple difference vectors; comparing therespective multiple difference vectors with corresponding baselinedifference vectors determined in order to train the multiple trainedartificial intelligence models; and classifying the given multimediasample as genuine or altered using at least the comparison.
 2. Themethod of claim 1, wherein the multiple trained artificial intelligencemodels comprise first and second trained artificial intelligence modelsand the method further comprises: training a first type of artificialintelligence system for a first modality of media to produce the firsttrained artificial intelligence model; and training a second type ofartificial intelligence system for a second modality of media to producethe second trained artificial intelligence model.
 3. The method of claim1, wherein a first of the multiple modalities comprises visual media anda convolutional neural network is used as the artificial intelligencemodel for this modality, and a second of the multiple modalitiescomprises audio or text and a recurrent neural network is used as theartificial intelligence model for this modality.
 4. The method of claim1, wherein the testing multiple trained artificial intelligence modelsfurther comprises recreating, using a dataset of genuine and alteredmultimedia content, both the genuine and altered multimedia contentusing respective first and second type of trained artificialintelligence models, the recreating performed until respectivereconstruction errors meet respective tolerance thresholds.
 5. Themethod of claim 1, further comprising outputting indication of theclassification of the given multimedia sample as genuine or altered. 6.The method of claim 2, further comprising adjusting model selectionparameters for the first and second trained artificial intelligencemodels using parameters specific to individual domains into which datafor a respective modality falls, in order to determine whicharchitectures work better for types of data in the modalities than otherarchitectures for those types of data.
 7. The method of claim 2, furthercomprising using the first and second trained artificial intelligencemodels on first and second difference vectors, formed usingcorresponding components during respective training of the first andsecond types of artificial intelligence systems, to create a combinationframework, wherein the combination network forms unified vectors fromthe first and second difference vectors.
 8. The method of claim 4,wherein: the comparing further comprises comparing respective multipledifference vectors with corresponding baseline difference vectors thatare mean vectors for respective modalities determined during training,and the comparing creates input difference vectors for respectivemodalities; and the classifying further comprises combining the inputdifference vectors for respective modalities into a unified input vectorand classifying the unified input vector as genuine or altered using aclassification framework that classifies based on unified differencevectors.
 9. The method of claim 7, further comprising recreating, usinga dataset of genuine and altered multimedia content, both the genuineand altered multimedia content using respective first and second type ofartificial intelligence systems, the recreating performed untilrespective reconstruction errors meet respective tolerance thresholds.10. The method of claim 9, further comprising: dissecting, after thereconstruction, internal layer outputs of the first and second trainedartificial intelligence models to determine which components of themodels to use to create respective difference vectors, wherein thecomponents comprise one or more of nodes or layers in respective models;comparing first and second difference vectors for the components in therespective first and second trained artificial intelligence models forgenuine and altered multimedia content; and deriving a set ofclassifiers from the first and second difference vectors based on thegenuine and altered multimedia content.
 11. An apparatus, comprising:one or more memories having computer-readable code thereon; and one ormore processors, the one or more processors, in response to retrievaland execution of the computer-readable code, causing the apparatus toperform operations comprising: testing multiple trained artificialintelligence models using known genuine samples of respective multiplemodalities of multimedia to generate versions of the multiple modalitiesof a given multimedia sample, wherein data for the multimedia and themultimedia sample are divided into the multiple modalities; computing,based on the testing, respective differences between respectivecomponents of the multiple trained artificial intelligence models toproduce respective multiple difference vectors; comparing the respectivemultiple difference vectors with corresponding baseline differencevectors determined in order to train the multiple trained artificialintelligence models; and classifying the given multimedia sample asgenuine or altered using at least the comparison.
 12. The apparatus ofclaim 11, wherein the multiple trained artificial intelligence modelscomprise first and second trained artificial intelligence models andwherein the one or more processors, in response to retrieval andexecution of the computer-readable code, further cause the apparatus toperform operations comprising: training a first type of artificialintelligence system for a first modality of media to produce the firsttrained artificial intelligence model; and training a second type ofartificial intelligence system for a second modality of media to producethe second trained artificial intelligence model.
 13. The apparatus ofclaim 11, wherein a first of the multiple modalities comprises visualmedia and a convolutional neural network is used as the artificialintelligence model for this modality, and a second of the multiplemodalities comprises audio or text and a recurrent neural network isused as the artificial intelligence model for this modality.
 14. Theapparatus of claim 11, wherein the testing multiple trained artificialintelligence models further comprises recreating, using a dataset ofgenuine and altered multimedia content, both the genuine and alteredmultimedia content using respective first and second type of trainedartificial intelligence models, the recreating performed untilrespective reconstruction errors meet respective tolerance thresholds.15. The apparatus of claim 11, wherein the one or more processors, inresponse to retrieval and execution of the computer-readable code,further cause the apparatus to perform operations comprising: outputtingindication of the classification of the given multimedia sample asgenuine or altered.
 16. The apparatus of claim 12, wherein the one ormore processors, in response to retrieval and execution of thecomputer-readable code, further cause the apparatus to performoperations comprising: adjusting model selection parameters for thefirst and second trained artificial intelligence models using parametersspecific to individual domains into which data for a respective modalityfalls, in order to determine which architectures work better for typesof data in the modalities than other architectures for those types ofdata.
 17. The apparatus of claim 12, wherein the one or more processors,in response to retrieval and execution of the computer-readable code,further cause the apparatus to perform operations comprising: using thefirst and second trained artificial intelligence models on first andsecond difference vectors, formed using corresponding components duringrespective training of the first and second types of artificialintelligence systems, to create a combination framework, wherein thecombination network forms unified vectors from the first and seconddifference vectors.
 18. The apparatus of claim 14, wherein: thecomparing further comprises comparing respective multiple differencevectors with corresponding baseline difference vectors that are meanvectors for respective modalities determined during training, and thecomparing creates input difference vectors for respective modalities;and the classifying further comprises combining the input differencevectors for respective modalities into a unified input vector andclassifying the unified input vector as genuine or altered using aclassification framework that classifies based on unified differencevectors.
 19. The apparatus of claim 17, wherein the one or moreprocessors, in response to retrieval and execution of thecomputer-readable code, further cause the apparatus to performoperations comprising: recreating, using a dataset of genuine andaltered multimedia content, both the genuine and altered multimediacontent using respective first and second type of artificialintelligence systems, the recreating performed until respectivereconstruction errors meet respective tolerance thresholds.
 20. Theapparatus of claim 17, wherein the one or more processors, in responseto retrieval and execution of the computer-readable code, further causethe apparatus to perform operations comprising: dissecting, after thereconstruction, internal layer outputs of the first and second trainedartificial intelligence models to determine which components of themodels to use to create respective difference vectors, wherein thecomponents comprise one or more of nodes or layers in respective models;comparing first and second difference vectors for the components in therespective first and second trained artificial intelligence models forgenuine and altered multimedia content; and deriving a set ofclassifiers from the first and second difference vectors based on thegenuine and altered multimedia content.
 21. A computer program productcomprising a computer readable storage medium having programinstructions embodied therewith, the program instructions executable byan apparatus to cause the apparatus to perform at least the following:testing multiple trained artificial intelligence models using knowngenuine samples of respective multiple modalities of multimedia togenerate versions of the multiple modalities of a given multimediasample, wherein data for the multimedia and the multimedia sample aredivided into the multiple modalities; computing, based on the testing,respective differences between respective components of the multipletrained artificial intelligence models to produce respective multipledifference vectors; comparing the respective multiple difference vectorswith corresponding baseline difference vectors determined in order totrain the multiple trained artificial intelligence models; andclassifying the given multimedia sample as genuine or altered using atleast the comparison.